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(54) Detecting scene cuts in video processing. 

(57) A method for detecting scene cuts in a motion picture, which comprises a plurality of frames in a 
sequential order. The preferred embodiment of the present invention utilizes the correlation between 
successive frames to detect abrupt cuts. Gradual cuts are preferably detected by examining a function 
consisting of the standard deviation of the pixels of each frame as a function of the frame number. 
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The present invention relates to video processing and more particularly, to an improved method for auto- 
-X^o^ 

« ramera Dans dissolves, fades to black or fade from black. 

oTLTod for summarizing a motion picture is to provide a log of the scenes. A scene og . a ,,t of h 
scenes and the location of the start of the scene from some reference point such as the start of the mot,on 
oicture A typical frame from the scene may be used to summarize the scene. 

Among other things, such logs are useful in video editing. The log provides a means for an ed.tor to qu.ckly 

' PrioTartthemeTior automating the logging of motion pictures rely on the difference between sequential 

cut has occurred. These methods compute the average value of some parameter 
J^^Zl value of the difference of corresponding pixels in two sequential frames. Such me hods 
do notdltect gradual cuts with sufficient reliability to be useful in automating the cut logging process. In ad- 
drtion there a?e some situations in which even abrupt cuts are not detected by these methods. 

Broad v " ISoJU of the present invention to provide an improved method for detectmg scene cuts 
II I further ^objecfof the present invention to provide an automated method for detectmg both gradual 

""rcoXrtheinvention proposes a method for detecting scene cuts in a motion picture, said motion 
oicture coTpling an ordered sequence of frames, each said frame comprising a plurahty of intensity values, 
sal metho Tcompiing the steps of determining a correlation value of each frame with a corresponding pre- 
vtous frame said prevfous frame being separated in said sequence by a predeterm.ned numbe °f frames, 
comparing «E ^relation value with a threshold value; and reporting an abrupt scene change ,f sa.d corre- 

detect abrupt cuts Gradual cuts are preferably detected by examining a function consist.ng of the standard 
dSion of the Pixels of each frame as a function of the frame number. Gradual cuts are characterized by 
S^^M^^V** scene cut may either manifest itself by a peak in this function or by a 

ramp connecting two regions of relatively constant values. a motion 

Figure 1 is a flow chart of an exemplary method to the present invention for f ind.ng the cuts in a motion 

PiCt The oresent embodiment utilizes the correlation of the pixels in successive frames to detect abrupt scene 
. eut.^n^CSTmom gradual scene cuts are detected will be explained in more detail ^ ow^Denote 
^Toilers^Tnth frame of a motion picture by ".(x,y). That is ".(x,y) is the intensity of the pixel at some po n 
x vfS h f ame FoTthe purposes oJthis discussion, it will be assumed that m(x,y) is defined on a regular 
grid having a total of ^4 grid points. For two frames, (n-k) and n, the method of the present mvention computes 
a correlation factor, p n , k , where 

EF n - k >l(x,y) * n «x,y)1 - Ef(" ^ k J|fx.v)1 * Epl(x.y)] (1) 

Pn ' k " C7 n * Cr (n -1) 

where 

E[('I(x,y)]4lI"Kx,y) (2) 

and . 

o n = \E["|2(x,y)l - E2[n|(x,y)l (3) 
In the oreferred embodiment of the present invention, k=1 . Within any given scene, p,,, is expected to be near 
^e IfTn aXtTcene cut occurs at frame n, then Pn ,, decrease to near zero. The present invention t acks 
°he value of Pn 'from frame to frame and defines an abrupt scene cut as having occurred at frame n ,f p., is 

^'Tn SSoTr^Hu. may be found by examining a few scene changes. However, it will be ap- 
nJnt to those IkniedTthe art that fixed thresholds may be used. The cut-off can also be defined in terms 
^^^^^ of Pn ., in such a system, an abrupt scene cut would be define he , have , oc- 
curred atframe y n if Pn , is less than the average value of Pn , over the previous M frames where M ,s some 
integer greater than 0. 
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While the above method finds abrupt scene cuts with a high degree of reliability, it is not a reliable method 
for finding gradual scene cuts, since the amount of change between successive frames in a gradual cut is too 
small to substantially change the value of p n(1 . In principle, the method discussed above can be extended to 
find gradual scene cuts by increasing the "distance" between the frames for which P n ,k is computed, i.e., choos- 

5 ing k>1 . In such a system, abrupt cuts would be detected using p M , and gradual cuts would be detected using 
p 0ik with k set to some predetermined value. While such a scheme is capable of detecting more of the gradual 
cuts, it requires some knowledge of the correct value for k. In addition, further analysis is needed to determine 
" the precise location of the cut. 

In the preferred embodiment of the present invention, an alternative method is used to detect gradual cuts. 

10 As discussed above, the computation of p n , k requires the computation of a n . The preferred embodiment of the 
' present invention uses the behavior of a n to detect gradual scene cuts. The value of a n for any frame measures 

the variation of the pixel values about the mean pixel value in the frame. Within any given scene, the values 
of a n from frame to frame remain roughly constant. When a scene slowly changes, the value a n changes from 
the value representative of the previous scene to the value representative of the new scene. This change takes 

15 place over a number of frames. 

The specific manner in which a n changes will depend on the starting and ending scene, and the nature 
of the special effect. For example, in the case of a fade to black followed by a fade from black to the new scene, 
cr n would be expected to decrease during the fade to black and then increase to the value representative of 
the new frame. In practice, the a n curve has considerably more noise than the p n , k curve; hence, the curve is 

20 smoothed in the preferred embodiment of the present invention to allow automated determination of the gradual 
cuts. In the preferred embodiment of the present invention, the previous 4 points are averaged to generate 
the smoothed version of a n . 

The excursions of the a n curve during a gradual scene cut are of two possible types. In the first type, the 
<7 n curve goes through a peak. This is similar to the fade to black effect discussed above. In the second type, 

25 the a n curve exhibits a "ramp" connecting the two regions of fairly constant a n values. 

A gradual cut is defined as having begun when the absolute value of the slope of the smoothed a n curve 
increases beyond a predetermined threshold value. The increased slope value must be present for a prede- 
termined first time interval, i.e., number of frames, before the gradual cut is regarded as valid. When the ab- 
solute value of the slope has remained below a predetermined threshold for a second time interval, the gradual 

30 cut is deemed to have been completed. Between the beginning and end of the gradual cut, the slope of the a n 
curve will, in general, go to zero briefly as the a n curve passes through an inflection point if a peak is present; 
hence, the gradual cut is not defined to have been completed until the absolute value of the slope of the a n 
curve remains below the threshold for some predetermined time interval. 

Refer now to Figure 1 which is a flow chart of a scene eut detecting program according to the present in- 

35 vention. In the preferred embodiment of the present invention, the value of k in Eq. (1) is set to one. To simplify 
the notation, p„ (1 has been replaced by p in the following discussion. After initializing various variables as shown 
at 10, the program acquires the next frame of the motion picture and computes the values for p and the average 
value of a as shown at 12. The program then tests the value of p to determine if an abrupt cut has occurred 
as shown at 14. An abrupt cut is deemed to have occurred if the value of p is less than a predetermined thresh- 

40 old. It is possible for an abrupt cut to take more than one frame. Hence, a flag is set as shown at 16 to indicate 
that p has crossed the threshold. The flag will be reset when p once again returns to a value above the thresh- 
old. This flag is used by blocks 18 and 20 to determine if the current value of p being less than the threshold 
is a new cut or just a continuation of the previously detected abrupt cut. If the cut is completed, the flag is reset 
as shown at 22. 

45 If no abrupt cut is in progress, the program goes on to examine the data to determine if a gradual cut is in 

progress. As noted above, the program uses the average value of a to detect gradual cuts. The curve consisting 
of the average value of ct as a function of the frame number will be referred to as the a curve. The program 
also keeps track of the slope of the a curve. Changes in the slope of the a curve are used to detect possible 
events in traversing a gradual cut. As noted above, a gradual cut may be evidenced either by a peak in the a 
: so curve or by a ramp connecting two a values representative of a in the frames on each side of the gradual cut. 

In the later case, there will be no change in the sign of the slope of the a curve during the gradual cut. 

Referring again to Figure 1, if no abrupt cut is in progress, the program tests for a gradual cut being in 
progress as shown at 24. If a gradual cut is not in progress, the program checks for the beginning of a gradual 
cut as shown at 26. A gradual cut is deemed to have begun if the slope of the a curve has changed and the 
55 change in a is greater than a predetermined threshold value. If these conditions are met, a set of flags is ini- 
tialized as shown at 28. The flag values indicate that a gradual cut has begun and no peak has yet been detected 
in the a curve. 

If a gradual cut was already in progress as indicated by the flags set at 28, the program determines if the 
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15 



20 



25 



curren, frame ,, .he end o, me gradua, cu, as ^ - 

the program checks to see if the program 1« ; •* "™EJ£ ie »he initial value is .«U pres- 
of .he above-described flags having a value different f r«n «s_ ™ in(|eeti0 „ poinI in „ curve. 

^r=rs:i r rirnr=^:rs^n,, y ^ ™. . e , 

in each frame for detecting gradual cuts ,t will be •%^*^^^ a is a function of the variance of 

ZZLZ one of the moments of the pixel dis ^ 

While the above-described embodiments of he J" 6 ^^ in tne art that a subset of the 

deviation using all of the data in each frame, it w,H be appa ^ <^ Jjj sybset may be used . Embodi- 

to those skilled in the art that special purpose hardware W^^f™^ J ti , e variance of 
operates. 
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Claims 
1. 



•„ o mntinn nirture said motion picture comprising an ordered se- 
A method for detecting scene cuts .n a T*"" inSnaity values, said method comprising the 
quence of frames, each said frame ^^""f^'^^^^^^ previous frame, said previous 
steps of determining a correlation value of each frame w.th " ~™P™ a " comparing sa id correlation 
frame being separated in said sequence by a predetermmeo ^ 

.relation value is less than 

value with a threshold value; and reporting an abrupt scene cnange 
. said threshold value. 

" 2 . - The method of Claim 1 wherein said predetermined number of frames is equal to one. 

3 The method of Claim 1 further comprising the steps of 

determining a statistical measure for each said frame ana and . 

4. The m.,hod of Claim 3 „h.re,n said «Hc. measure » a functfcn of .he variance of .aid in.ansi., vai- 

ues. 

» 5 The method of Claim 3 wherein said statistica. measure is a moment of said intensity values. 

6. The method of C.aim 3 wherein said predetermined pattern is a peak in said statistical measure as a func- 
tion of the frame number. 

ure between the beginning and end of said ramp. 
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(54) Detecting scene cuts in video processing. 

(57) A method for detecting scene cuts in a motion 
picture, which comprises a plurality of frames in 
a sequential order. The preferred embodiment 
of the present invention utilizes the correlation 
between successive frames to detect abrupt 
cuts. Gradual cuts are preferably detected by 
examining a function consisting of the standard 
deviation of the pixels of each frame as a 
function of the frame number. 
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